Cycle frequency in standard Rock-Paper-Scissors games: 
Evidence from experimental economics 



m 
o 

(N 
£ 

in 



Bin Xu 1 ' 2,3 and Zhijian Wang 2,3 * 
1 Public Administration College, Zhejiang Gongshang University, Hangzhou, 310018, China 
2 Experimental Social Science Laboratory, Zhejiang University, Hangzhou, 310058, China and 
3 State Key Laboratory of Theoretical Physics, Institute of Theoretical Physics, 
Chinese Academy of Sciences, Beijing, 100190, China 
(Dated: January 16, 2013) 

In the iconic game, standard Rock-Paper-Scissors (RPS) game, cycle in evolutionary trajectory is 
predicted by evolutionary game theory but not by classical game theory. We conduct lab experiments 
of the standard RPS game to index the cycle. Our experiments are of traditional setting with discrete 
time and random matched. In the 12 samples of 300 rounds 'stochastic' social evolution trajectories, 
(1) We confirm that the cycles not only exist but also persist; (2) We firstly test out the frequencies 
of cycles in this game; The observed frequency is about 0.028 ±0.007 period per experimental round. 
The application of the observed frequency on a dynamics model is illustrated. Using frequency of 
cycles to index the distance from equilibrium, we consider both of the time symmetry and the 
sampling size effect, both of which have the root in non-equilibrium statistical physics. 

PACS numbers: 89.65.-s 87.23.Cc 02.50.Le 01.50.My 
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INTRODUCTION 

In evolutionary trajectories, formulating a physically 
meaningful measure of the distance from equilibrium is 
itself an area of active research An equilibrium sys- 
tem satisfies detailed balance condition, in which time 
reversal symmetry holds and the distance from equilib- 
rium is zero. If a system is in non-equilibrium stationary 
state, the long-run balances of transits within states are 
broken and the distance from equilibrium should devi- 
ate from zero. Observations, e.g. entropy production Q 
and velocity Q, have been developed to index the dis- 
tance from equilibrium. This picture comes from non- 
equilibrium statistical physics. In this paper, this pic- 
ture is borrowed to investigate the trajectories from lab 
experiments in game theory. 

Game theory studies strategies interactions and social 
evolutionary trajectories in the systems from bacteria to 
human being ka-|Z| . In game theory, there exist two theo- 
ries — classical game theory (CGT) and evolutionary 
game theory (EGT) |{|. From the two theories, the pre- 
dictions are identical on Nash equilibrium, but different 
on the evolutionary trajectories. 

We use the standard Rock-Paper-Scissors (RPS) game 
as an example. This game is iconic in both CGT and 
EGT [iWlol]. In the evolutionary trajectories surround 
Nash equilibrium, in long run, EGT predicts the exis- 
tence of cycles, the system is in non-equilibrium and the 
transits within the states are not in detailed balance. Al- 
ternatively, CGT predict full random — the system is in 
equilibrium, in long run, each loop should be balanced 
by its reverse loop. Because, the system is in equilib- 



rium state, in which corresponding to every individual 
process there is a reverse process, and then the average 
rate of every process is equal to the average rate of its 
reverse process [0]; So CGT predicts that, in long run, 
excess loops (hereafter, called as cycles) cannot be ob- 
served in the trajectories. So, the crucial bifurcation of 
the two theories is whether the cycles are existent. Using 
cycles as an object, we will show that the distance from 
equilibrium can be measured, and then can be employed 
to evaluate evolutionary dynamics model. We carry out 
these tasks in lab experiments. 

Lab experiments study human subjects' strategy in- 
teractions and have been major tool to test game the- 
ory 
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In past decades, lab game experiments have 
been employed to test EGT [l3j], in which to report the 



cycles is a task |14| - |l6l | . The existence of cycles in the evo- 
lutionary trajectory has been supported only with figures 
qualitatively EH. Only quite recently, in RPS-like 
game, but in a special setting (continuous time) games, 
the existence cycles are firstly observed [13] • Regrettably, 
in traditional setting (discrete time and random match- 
ing [l3 - 15, I8I-I2H) game experiment environments, in the 
evolutionary trajectories, cycles has never been indexed 
quantitatively. 

Discrete time games, rooting in the natural period 
(e.g., the yea r for anim al |5| ), are concerned by exper- 
imenters |Tol [l3l-[l5l II8M21I] and theorists pi l^. Our 
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experiments are standard RPS games of discrete time. 
In these experiments, we harvest 12 experimental tra- 
jectories and each includes 300 rounds. Like in existed 
experiments [l4, 15, 18, 2(| 21 1, the evolutionary trajec- 
tories are high stochastic. Borrowing the picture from 
non-equilibrium statistical physics to measure the dis- 
tance from equilibrium (or detailed balance) in trajec- 
tories [24|, we index the cycles (excess loops surround 
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FIG. 1: Payoff matrix of the standard RPS Game. 



Nash equilibrium) in the stochastic evolutionary trajec- 
tories quantitatively. 

In our discrete time standard RPS game experiments, 
we confirm the cycles exist, persist and do not dissipate; 
Then, we test out the observed frequency of the cycles 
is about 0.028 (±0.007) period per experimental round. 
Then, we demonstrate how to evaluate an EGT model 
with the observed frequency. 

This paper is organized as following. In next session, 
we introduce the traditional setting standard RPS exper- 
iment. In section 3, the measurement methods are intro- 
duced; In section 4, we report the performance of the 
cycles and the support materials; Section 5 is to demon- 
strate also how to evaluate models with frequency. Dis- 
cussion and summary is in the last. 



EXPERIMENT 

In RPS game, rock (R) beats scissors (S) which beat 
paper (P) which beats rock. Standard RPS means, tak- 
ing tic as reference, the benefit for winner is equal to loss 
for the opponent. This game is simplest and iconic for 
both CGT and EGT [13, 1H ill • The payoff matrix used 
in the experiment is presented in Fig. [TJ One will get 2 
experiment points if win, 1 if tie and if loss. 

The laboratory experiments consist of 12 independent 
groups. Each group consists of 6 players; so our experi- 
ment is a finite population (the population size is 6) game 
experiment. Each group plays the RPS game 300 rounds 
(the aim of this 300- round setting should be discussed fol- 
lowing). In each experimental round, every player plays 
the game against a opponent who is randomly selected 
by computer program within a group. Such experimental 
set ting is a traditional in experimental economics [il- 
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All subjects make their own decisions simul- 
taneously and anonymously. Only after all subjects sub- 
mitted their choices, each of them would get the feedback 
including her/his strategy, her/his opponent's strategy 
and her/his own point through their private screen. No 
other information was provided. And these are common 
knowledge. 

The 12 experimental sessions were conducted during 
Dec. 2010 in experimental social science laboratory, Zhc- 



jiang University. All 72 subjects were recruited broadly 
from the student populations at Zhejiang University. 
They were sitting in an isolated seat with a computer 
after arrived. Both written and oral instructions were 
provided for every subject before experiment implement- 
ing. During the experiment, subjects gained experiment 
points each round according to the payoff matrix. The 
experimental sessions were lasted about 1.5 to 2 hours. 
The subjects got the earnings in cash privately after the 
experiments according to the accumulated points over 
300 experimental rounds. The exchange rule is 1 exper- 
iment point equals 0.15 Yuan RMB. In addition, each 
subject get 5 Yuan RMB as show-up fee. The average 
earning is 50 Yuan RMB. 



NOTIONS AND MEASUREMENT 
Notions 

Social state Social state has been used to repre- 
sent social performance in game widely [1(1 26]. In evolu- 
tionary game theory view, the social state is described by 
the densities of strategies in a society. In RPS game, the 
strategy set is (S,R,P), so the social state is denoted 
as (x,y,z), herein, x,y,z are the densities of strategy 
Scissors, Rock and Paper in a society respectively. Since 
z = 1 — x — y, the state can be represented in simply as 
(x, y). If there are six persons in a group (society) just 
as our experimental setting, the total number of state is 
28. 

State space All potential states compose the 

state space. In the case of our six person RPS game, the 
state space is an equilateral triangle including 28 states 
in a two dimensional plane. We illustrate the state space 
and 28 social states in Fig. [2l For example, state (0, 0) 
in the bottom left represents there arc scissors, rock 
and 6 paper in the group, and the state (1/3, 1/3) in the 
center represents 2 scissors, 2 rock and 2 paper in the 
group. 

Evolutionary trajectory Evolutionary trajec- 
tory is the evolutionary trace of social state transit in 
state space 
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In our experiments, each 
six person group repeated the game 300 times. For each 
round, the social state should be in and only in one of the 
discrete 28 states. From round t to t + 1, the social state 
may change and we call this change as transit. The social 
state should transit 299 times during an experiment and 
form an evolutionary trajectory. 

Cycles in a trajectory In the view of evolu- 
tionary game theory, the evolutionary trajectory in RPS 
game here should form counter-clockwise cycles 0, 
|25| . However, the evolutionary trajectory is full of noises 
and we can hardly test out robust pattern, especially in 
short run. Strictly speaking, in our experimental tra- 
jectories, the cycles are the observed loops of transits 
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FIG. 2: State space and Poincare section. There are total 28 
observable states in six person RPS game. The state space is 
an equilateral triangle including 28 states in two dimensional 
plane, in which, state (0, 0) in the bottom left represents there 
are scissors, rock and 6 paper in the group, and the state 
(1/3,1/3) in the center represents 2 scissors, 2 rock and 2 
paper in the group. The segment S is the Poincare section for 
cycles counting. In our measurement, the segment is shifted 
(to left-down in 1CP' ! ) slightly and it does not pass through 
any of the 28-state. 



surround N e in one direction excess over the opposite 
direction (EH SI]. 

Distance from equilibrium One point need 

to be clarified. The distance from equilibrium in this 
paper is to index the asymmetry, or unbalance, of the 
transitions within states in trajectories. As mentioned, 
our distance from equilibrium is indexed by the cycles 
(the excess loops) in the trajectory. On the contrast, in 
game experiment literatures, e.g., [13, E3 which are most 
close to this investigate, the distance from equilibrium, 
called as the distance from the center [id 171, is to index 



the distance from an observed state to a Nash equilibrium 
point. 



Measurement 



Poincare section and cycles 



— Following 

Ref 24 using excess transitions to represent the deter- 
ministic process in stochastic processes, we further use 
cycle counting method to represent the cycles and take it 
as the index of distance from equilibrium. Poincare sec- 
tion which introduced into experimental economics by 
Cason et al [ItJ is an effective tool to count cycles. As 
illustrated in Fig. [3J a line segment is constructed in the 
state space from N e to one edge of the triangle. Accord- 
ing to dynamics models and the evolutionary stable state 
concept in EGT 0, [Io|, 25], we postulate that, if cycles 
exist, they should surround the center — the Nash equi- 
librium denoted as N e at (|, ^). As mentioned above, 
each group would have a 300 rounds trajectory including 
299 transits in state space. Some of the transits cross the 



Poincare section (S) from left to right (counter-clockwise) 
and some from right to left (clockwise). 

We denote every transit-i as Ci, assign the transit- i 
crossing S as Ci — 1 (-1) when the transit is counter- 
clockwise (clockwise) and assign the transit-i not cross- 
ing S as Ci = 0. Then, we defined the number of the 
counted cycles in time interval [to, ^i] as 



a 



to,ti 



(i) 



If the evolutionary trajectory is full of random tran- 
sits as expected by CGT, in long-run, C e should be zero, 
otherwise, if C e > significantly, there should exist dis- 
tance from equilibrium. Mathematically, C e is of time 
odd Cf l to = — tl , and will be discussed below. 

Frequency of cycles To compare the cycles 

in different length of game, we adopt frequency as an 
index. Frequency denoted as f e means the cycles in per 
experimental round and is calculated as 



fe 



ti — to 



(2) 



This observable quantitatively captures the distance from 
equilibrium at one instantaneous snapshot in the experi- 
mental trajectories. The physical mean of the frequency 
is, on average, how many cycles can be obtained in each 
experimental round. 



RESULTS 

In all 12 300-rounds experiments, the strategies S, R 
and P have been used 6961, 7702 and 6937 times respec- 
tively. On average, the proportion of strategy S, R and 
P is 0.322, 0.357 and 0.321 respectively. This result is 
close to Nash equilibrium (1/3, 1/3, 1/3). 

There are total 12 trajectories obtained from the exper- 
iments. Indeed, the experimental trajectories are highly 
stochastic. This result is the same as the observations 



in other experiments [14j, [15|, [Ha, |20|, [21[ . Our main task 



is to test the cycles in the 12 experimental trajectories. 
The results are as follows. 

Frequency of cycles Using Eq. [TJ we calculate 

Cf 300 . For each of the 12 experimental trajectories, the 
numerical results of Cf 300 are shown in the last column 
in Tabled] 

On average, over the 12 experimental trajectories, 
Cf 300 — 8.50 ± 2.17. Accordingly, the observed frequency 
of cycles (/ e ) is 0.028 ± 0.007. That is to say, the em- 
pirical frequency of the cycles is 0.028 ±0.007 period per 
experimental round. And the 95% confidence interval is 
[0.012 0.044]. 

Existence of cycles To estimate the existence 

of the cycles, we use Cf 300 from the 12 group samples. 
Statistical result shows that Hi: Cf 30 o=0 can be rejected 
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TABLE I: C e in the 12 trajectories 
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The last three rows are the statistical results of the 12 
experimental groups. The last row is p-value of the t-test by 
setting the null hypothesis C\ t of the 12 samples above. 
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FIG. 3: Counted cycles Cf >t of the 12 sessions with t = 5, 
10, 15, . . ., 300. For example, ending at +23 at t = 300, 
the blue curve (the 7-th group in the 12 groups) reads 23 
counter-clockwise cycles. 



(p = 0.002 < 0.05, t-test, two-tailed, d.f-ll). So, we 
can say that cycles do exist in the standard RPS game 
in discrete time experimental setting. 

Direction of cycles Statistical result also shows 

Ci 300 > (p=0.001), that means the direction of cycles 
are of counter-clockwise. This result meets the direction 
expected by variety evolution dynamics (j| [l(| HH . Set- 
ting t = 5, 10, 15, 300, the Cf t of each group are ex- 
hibited in Fig. [3] Nine out of twelve groups the counted 
cycles are positive, two groups are zero and one is nega- 
tive; Statistically, the C e is positive (p < 0.01, ttest with 
9 sample of +1, 2 samples of and 1 sample of -1) and 
the direction is of counter-clock. 

Persistence of cycles To see the persistence 

of cycles, we set the null hypothesis as H 2 - Cf 150 = 
6451300- If H2 can be rejected in the data, cycles do 
persist, which mean EGT is supported and CGT is de- 
clined and vice versa. We use the p-values from t-test 
of the null hypothesis (the dissipation of cycles) H^'- 



^1150 = ^151 300- Statistical result is p > 0.10. So, 
in our 12 300-rounds trajectories, the dissipation of cy- 
cles in the data can be rejected. In another word, the 
persistence of the cycles cannot be rejected by the data. 

Accordingly, this result is our answer to the question 
'Do cycles dissipate when subjects must choose simulta- 
neously' raised by the authors 17 1. 

Length of the experimental round to confirm 

cycles One result need to be emphasized. We have 

test, at what value of t, the null hypothesis H±: Cf t =0 
can be rejected. According to the last row in Table U we 
find that, the p-values are below 0.05 only when t > 150. 
This means that, using the 12 trajectories, to confirm 
the existence of cycles, the length should be at least 150 
rounds. This point relates to the marginal value to test 
out cycles and will be discussed below. 



MODEL EVALUATING 

Till now, using frequency of cycles to index the dis- 
tance from equilibrium, we have reported the empirical 
results from the experiments data. As evaluating param- 
eters of dynamics models is a task in EGT investigation 
(e.g., in this section, we apply the frequency of cy- 

cles, as a novel way, to evaluate parameters in EGT. 

For simple and referring to the experimental literatures 
to evaluate EGT [TMl, we use the continuous time and 
infinite population size approximation. Here, the demon- 



stration is to evaluate the parameter in logit dynamics 
model in EGT. This dynamics works as follows: at every 
stage of the game players are selected uniformly at ran- 
dom according to a noisy best-response where the noise 
level is tuned by a parameter /3 [25|, [28[ . This parameter 
is a linking from the temperature in statistical physics to 
the degree of the rationality of human agents in strategy 
interaction (l9l. I29I - I32I ] . Explicitly, the logit dynamics can 
be presented as, 



exp (fiUj) 
E,cxp(/3ig 



(3) 



in which Ui is the mean payoff of the agents using strategy 
i. The payoff-matrix of the game is in Fig. [T] At state 
[x, y, z), the mean payoff Ui of the three strategy can be 
presented as 



(4) 



It is not surprise that, the mean payoff (U = xU x + 
yUy + zU z ) of this population always 1; Because the stan- 
dard RPS game is a constant sum game and sum of each 
pair players are 1. 

According to Eq. 01 the logit dynamics equation 
(Eq. [3]) can be specified to 



'u x ~ 
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1. Time reversal symmetry 



As the central 
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(5) 



in which X= [e t3 < 2 x ^ + e 13 + f 2 -*- 2 »>] . As 

the density strategy of Scissors (z) always determined 
when the density strategy of Rock (a;) and Paper (y) are 
given (z = 1 — x — y), z is determined by x and y in Eq. [5] 
and can be ignored. 

At a equilibrium state, the velocity of the strategy 
change should be 0. Then, setting the right-hand-side 
of Eq.[5]as 0, solving the equations, the equilibrium state 
can be obtained. That is to say, for a given j3, we can 
obtain the mixed strategy equilibrium. No surprising, for 
any given /3, (1/3, 1/3, 1/3) is always the solution and is 
the equilibrium state. 

Surround the equilibrium state, the Jacobian is the 
function of /3 and can be expressed explicitly as 



-1/3-1 -§/3 
1(3 



From which, the eigenvalue e of this Jacobian is 



(6) 



(7) 



As the frequency of cycles is determined by the image 
part of the eigenvalue divided by 2ir, so theoretical fre- 
quency of cycle ft ((3) = (3. 

Using the empirical frequency (/ e ), we have the best 
fit (ft = fe) for the model parameter (3 is, 



/3 = 0.305 ±0.076 



(8) 



and the 95% of C.I. of /3 is [0.131 0.479]. This param- 
eter could index an experimental population (e.g., the 
degree of rationality [30j). The fixed parameter (3 in 
Eq. p could links to fundamental models in game the- 
ory [2ll l3(il ] and free energy like observations [jj . In this 
section, we have demonstrated that experiment test of 
non-equilibrium game behavior is possible. 



DISCUSSION 

In a standard RPS experiment with discrete time set- 
ting as I18M21I ] , we found the cycles exist and per- 
sist. Firstly, we test out the frequency of cycles in discrete 
time RPS game. We have also demonstrated how to fix 
the model parameter using the empirical frequency. 

Comparing with existed experiments on EGT testing, 
we suggest that, our finding base on the two advantage 
points: (1) We have considered the time reversal asym- 
metry in measurement. (2) We have considered the sam- 
pling size effect in our experiment. 



measurement of this report, C e in Eq.(fT|) is of time 
odd (C^ t 2 =-C t e 2 tl ). In this way, the cycles be- 
yond detailed balance can be counted. One can 
notice that, for the cycles counting in a stochastic 
trajectory, this measurement has its root in non- 
equilibrium statistical physics (e.g., see the observ- 
able N + - A_ in [H]). 

Referring to existed exp eriment literatures, as em- 
phasis by reference [lTl [33| , cycle in evolutionary 
trajectory has being sought for long time. Till now, 
in discrete time game, the existence of cycles is only 
supported qualitatively 13, 3 IH but not quanti- 
tatively. 

Recently, two time odd observations, velocity vec- 
tor [26] | and angular momentum [34| , are developed 
to indexed the experimental evolutionary trajec- 
tory. Both of the observations can test out EGT 
patterns in significant. No surprising, these time 
odd observable, like entropy production 24. 35-37|. 
can capture the excess transits beyond detailed bal- 
ance and can index the distance from equilibrium 
in games. 



2. Sample size effect 



To test out the de- 



terministic behaviors, in higher stochastic system, 
longer experimental rounds and groups numbers 
is needed. It is sample size effect. This picture 
has been demonstrated in the experiment in non- 
equilibrium statistical physics (H . For example, the 
Fig. 1 in ref. (3, using 540 experimental trajecto- 
ries, the authors compare two time interval condi- 
tions (10 -2 sec and 2 sec with sampling At = 10~ 3 
sec). For 10 -2 sec, the probability of a trajectory 
and its time reversal trajectory seems nearly equal. 
On the contrast, the asymmetry in 2 sec samples 
can be test out significantly. In experimental game 
trajectories, the condition is similar. 

As large sample size are appropriate, we set the ex- 
perimental round up to 300 rounds. According to 
the statistical result in our data, Cf t > could be 
significant only when t > 150 (see the last para- 
graph in the Result section). That is to say, even 
using 12 experimental trajectories, if the experi- 
mental rounds are less than the margin, we cannot 
confirm the existence of cycles even though time 
odd measurement has been used as Eq. [TJ That is 
also the reason why we set our experiments up to 
300 rounds. 

Referring to related discrete time experiments on 
EGT, the number of rounds and the groups usually 
are less. For example, in the experiments the 
experimental rounds is 100 and the tested groups is 
4 for a given parameter; As a result 17| , cycles can- 
not be test out. Usually, the experimental trajec- 
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torics are less than 12 and the experimental rounds 
are less than 100 in a trajectory, from these experi- 
ments, to confirm the existence of cycles should be 
uneasy. 

Accordingly, we suggests that, for further inves- 
tigating the cycles in experiments, longer experi- 
ments rounds and larger experimental trajectories 
are expected. Only when the sampling size is large 
enough, excess observations beyond detailed bal- 
ance can be statistically significant. 

It can be noticed that both of the two points are rooting 
in non-equilibrium statistical physics and for measuring 
the distance from equilibrium. 

Linking experimental economics game to principles in 
statistical physics is developing, e.g., maximum entropy 
principle governs Nash equilibrium |38| and governs so- 
cial transits (39|; In lab game experiments 36 L 37|, fluc- 
tuation relations (Fluctuation Theorem) )24l. l40j is sup- 
ported. Evidences from experimental economics promote 
that there could exist wider common background of so- 
cial science and physical science [41 1. 



SUMMARY 

In standard RPS game, surround Nash equilibrium, 
EGT predicts cycles; Alternatively, CGT predict no cy- 
cle exists and the evolutionary trajectories stochastic. 
Results from our traditional setting (discrete time and 
random matching) experiments show that, the evolution- 
ary trajectories are high stochastic indeed, however, sur- 
round Nash equilibrium, cycles exist and persist. Inter- 
estingly, we test out the frequency of cycles. As demon- 
strated, the empirical frequency could be linked the evo- 
lutionary dynamics model. Our measurement has the 
roots in non-equilibrium statistical physics. 

We wish, basing on game theory and non-equilibrium 
statistical physics, the co-playing of lab experiments and 
dynamics models could lead us to see something what we 
have not seen yet. 
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